In practice, by simply training a generator and a discriminator together consisting of multi-layer neural networks with non-linear activation functions, using local search algorithms such as stochastic gradient descent ascent (SGDA), the generator network can be trained efficiently to generate samples from highly-complicated distributions (such as the distribution of images). Despite the great empirical success of GAN, it remains to be one of the least understood models on the theory side of deep learning. Most of existing theories focus on the statistical properties of GANs at the global-optimum [15, 16, 20, 87]. However, on the training side, gradient descent ascent only enjoys efficient convergence to a global optimum when the loss function is convex-concave, or efficient convergence to a critical point in general settings [37, 38, 48, 53, 71, 73, 75, 77, 78]. Due to the extreme non-linearity of the networks in both the generator and the discriminator, it is highly unlikely that the training objective of GANs can be convex-concave. In particular, even if the generator and the discriminator are linear functions over prescribed feature mappings-- such as the neural tangent kernel (NTK) feature mappings [3, 8, 9, 17, 18, 32, 35, 40, 41, 47, 51, 54, 65, 69, 92, 97] -- the training objective can still be non-convex-concave.

The composer classification question has been posed for a variety of corpora, from Renaissance composers [2,3], to the narrow (and challenging) case of Haydn and Mozart string quartets [5, 8, 12, 22], and to various collections of classical era composers (most of the other papers discussed in Section 2). In this work we study an expansive collection of scores, from 13th century sacred music by Guillaume Du Fay to 20th century ragtimes by Scott Joplin. A major challenge of this task is learning from limited data. While the corpus considered here is larger than most, this is largely due to the number of composers considered (19): for specific composers, we have at most 466 scores (Bach) and as few as 22 (Japart). Small datasets are an inherent problem for composer classification: the corpus used in this work contains, for example, all of the Bach chorales and all of the Mozart string quartets. We cannot resurrect these composers and have them write us more scores to include in our corpus. This situation contrasts starkly with many learning problems, where substantial progress can be made by collecting massive datasets and exhaustively training an expressive model (usually a deep neural network) with "big data."